Method for performing region-of-interest-based depth detection with aid of pattern-adjustable projector, and associated apparatus

ABSTRACT

A method for performing region-of-interest (ROI)-based depth detection with aid of a pattern-adjustable projector and associated apparatus are provided. The method includes: utilizing a first camera to capture a first image, wherein the first image includes image contents indicating one or more objects; utilizing an image processing circuit to determine a ROI of the first image according to the image contents of the first image; utilizing the image processing circuit to perform projection region selection to determine a selected projection region corresponding to the ROI among multiple predetermined projection regions, wherein the selected projection region is selected from the multiple predetermined projection regions according to the ROI; utilizing the pattern-adjustable projector to project a predetermined pattern according to the selected projection region, for performing depth detection; utilizing a second camera to capture a second image; and performing the depth detection according to the second image to generate a depth map.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates to detection mechanism for electronicdevices such as mobile devices, and more particularly, to a method forperforming region-of-interest (ROI)-based depth detection with aid of apattern-adjustable projector, and associated apparatus such as aROI-based depth detection device, an image processing circuit within theROI-based depth detection device, a depth processor within the imageprocessing circuit, the pattern-adjustable projector within theROI-based depth detection device, etc.

2. Description of the Prior Art

According to the related art, face recognition technologies have beenapplied to mobile devices, and some face recognition methods for use ofthe mobile devices have been proposed. However, some problems may occur.For example, when one or more of the mobile devices cannot perform facerecognition correctly, a security issue is introduced. There may be abottleneck of further improvement of the face recognition algorithms.Hence, there is a need for a novel method and associated architecture toenhance the overall performance of an electronic device withoutintroducing a side effect or in a way that is less likely to introduce aside effect.

SUMMARY OF THE INVENTION

It is therefore an objective of the present invention to provide amethod for performing region-of-interest (ROI)-based depth detectionwith aid of a pattern-adjustable projector, and to provide associatedapparatus such as a ROI-based depth detection device, an imageprocessing circuit within the ROI-based depth detection device, a depthprocessor within the image processing circuit, the pattern-adjustableprojector within the ROI-based depth detection device, etc., in order tosolve the above-mentioned problems.

At least one embodiment of the present invention provides a method forperforming region-of-interest (ROI)-based depth detection with aid of apattern-adjustable projector, where the method may comprise: utilizing afirst camera to capture a first image, wherein the first image comprisesimage contents indicating one or more objects; utilizing an imageprocessing circuit to determine a ROI of the first image according tothe image contents of the first image; utilizing the image processingcircuit to perform projection region selection to determine a selectedprojection region corresponding to the ROI among multiple predeterminedprojection regions, wherein the selected projection region is selectedfrom the multiple predetermined projection regions according to the ROI;utilizing the pattern-adjustable projector to project a predeterminedpattern according to the selected projection region, for performingdepth detection; utilizing a second camera to capture a second image,wherein the second image comprises a projection result of thepredetermined pattern on a first object among the one or more objects;and performing the depth detection according to the second image togenerate a depth map, wherein the depth map indicates a set of depths ofthe first object.

At least one embodiment of the present invention provides associatedapparatus operating according to the method mentioned above. Examples ofthe apparatus may include, but are not limited to: a ROI-based depthdetection device, an electronic product (e.g. an electronic device suchas a mobile device) equipped with the ROI-based depth detection device,the image processing circuit within the ROI-based depth detectiondevice, a depth processor within the image processing circuit, thepattern-adjustable projector within the ROI-based depth detectiondevice, etc.

At least one embodiment of the present invention provides an apparatusfor performing region-of-interest (ROI)-based depth detection, where theapparatus may comprise an image processing circuit, and further comprisea first camera, a second camera and a pattern-adjustable projector thatare coupled to the image processing circuit. For example, the firstcamera can be arranged to capture a first image, wherein the first imagecomprises image contents indicating one or more objects. The imageprocessing circuit can be arranged to determine a ROI of the first imageaccording to the image contents of the first image, and performprojection region selection to determine a selected projection regioncorresponding to the ROI among multiple predetermined projectionregions, wherein the selected projection region is selected from themultiple predetermined projection regions according to the ROI. Inaddition, the pattern-adjustable projector can be arranged to project apredetermined pattern according to the selected projection region, forperforming depth detection. The second camera can be arranged to capturea second image, wherein the second image comprises a projection resultof the predetermined pattern on a first object among the one or moreobjects. Additionally, the image processing circuit can perform thedepth detection according to the second image to generate a depth map,wherein the depth map indicates a set of depths of the first object.

The present invention method and associated apparatus (e.g. theROI-based depth detection device, the image processing circuit withinthe ROI-based depth detection device, the depth processor within theimage processing circuit, the pattern-adjustable projector within theROI-based depth detection device, etc.) can guarantee that variouselectronic products equipped with the ROI-based depth detection devicecan operate properly in various situations, respectively. In addition,the present invention method and associated apparatus can use thepredetermined pattern projected in a small transmitting (Tx) field lessthan a normal Tx field to increase the total power of the predeterminedpattern being projected, and use the projection result of thepredetermined pattern that has been projected according to the selectedprojection region corresponding to the ROI, to perform the depthdetection accurately and efficiently. In comparison with the relatedart, the present invention method and associated apparatus can enhanceoverall performance without introducing any side effect or in a way thatis less likely to introduce a side effect.

These and other objectives of the present invention will no doubt becomeobvious to those of ordinary skill in the art after reading thefollowing detailed description of the preferred embodiment that isillustrated in the various figures and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a ROI-based depth detection device according toan embodiment of the present invention.

FIG. 2 illustrates some implementation details of the pattern-adjustableprojector shown in FIG. 1 according to an embodiment of the presentinvention.

FIG. 3 illustrates, in the right half thereof, a small transmitting (Tx)field control scheme of a method for performing ROI-based depthdetection with aid of the pattern-adjustable projector according to anembodiment of the present invention, where a normal Tx field controlscheme is also illustrated as shown the left half of FIG. 3 for bettercomprehension.

FIG. 4 illustrates a ROI-based projection control scheme of the methodaccording to an embodiment of the present invention.

FIG. 5 illustrates some examples of multiple predetermined projectionregions used in the method.

FIG. 6 illustrates some other examples of the multiple predeterminedprojection regions used in the method.

FIG. 7 illustrates some examples of a location of an object with respectto a selected projection region among the multiple predeterminedprojection regions used in the method.

FIG. 8 illustrates some examples of multiple transitional projectionregions serving as a portion of the multiple predetermined projectionregions used in the method.

FIG. 9 illustrates some implementation details of the depth processorand the pattern-adjustable projector shown in FIG. 1 according to anembodiment of the present invention.

FIG. 10 illustrates a working flow of the method according to anembodiment of the present invention.

DETAILED DESCRIPTION

Embodiments of the present invention provide a method for performingROI-based depth detection with aid of a pattern-adjustable projector,and associated apparatus such as at least one portion (e.g. a portion orall) of a ROI-based depth detection device. The ROI-based depthdetection device can operate according to the method to perform depthdetection accurately and efficiently, to achieve optimal performance ofany electronic product equipped with the ROI-based depth detectiondevice. More particularly, the ROI-based depth detection device canperform the ROI-based depth detection accurately and efficiently togenerate one or more depth maps, such as depth maps of human faces, tomake the electronic product perform three-dimensional (3D) facerecognition according to the one or more depth maps accurately andefficiently. The electronic product can utilize the ROI-based depthdetection device to generate at least one depth map corresponding to atleast one user of the electronic product, for establishing a 3D facedatabase of the at least one depth map in advance, and can compare atarget depth map of a person's face among the one or more depth mapswith any depth map of the at least one depth map to determine whetherthis person is any user of the at least one user. If the target depthmap matches the any depth map (e.g. this person is the any user), theelectronic product can provide services to this person; otherwise (e.g.this person is not any of the at least one user), the electronic productcan prevent this person from using any of the services. Examples of theelectronic product may include, but are not limited to: a mobile devicesuch as a multifunctional mobile phone, a tablet, a wearable device, anall-in-one (AIO) computer, and a laptop computer.

FIG. 1 is a diagram of a ROI-based depth detection device 100 accordingto an embodiment of the present invention, where the ROI-based depthdetection device 100 and the pattern-adjustable projector 110 thereincan be taken as examples of the ROI-based depth detection device and thepattern-adjustable projector mentioned above, respectively. For bettercomprehension, a field 10 and one or more target objects 11 may beillustrated with reference to the X-axis, the Y-axis and the Z-axis onthe upper half of FIG. 1 , but the present invention is not limitedthereto. For example, the field 10 (e.g. the shape, the associated sizesand/or the associated angles thereof) and the one or more target objects11 (e.g. the object count, the shapes, the associated sizes and/or theassociated locations thereof) may vary.

As shown in FIG. 1 , in addition to the pattern-adjustable projector110, the ROI-based depth detection device 100 may further comprise animage-capturing module 120 and an image processing circuit 130, wherethe pattern-adjustable projector 110 and the image-capturing module 120are coupled to the image processing circuit 130. For example, theimage-capturing module 120 may comprise at least one camera (e.g. one ormore cameras) such as multiple cameras 121 and 122 respectivelycorresponding to depth-related processing and two-dimensional (2D) imageprocessing, and the image processing circuit 130 may comprise at leastone processor (e.g. one or more processors) such as a depth processor131 and an image processor 132 respectively corresponding to thedepth-related processing and the 2D image processing, but the presentinvention is not limited thereto. In some embodiments, the cameras 121and 122 may be integrated into the same camera, and/or the depthprocessor 131 and the image processor 132 may be integrated into thesame processor.

The pattern-adjustable projector 110 can be arranged to project one ormore predetermined patterns of invisible light (e.g. infrared (IR) orlights of other wavebands). For example, a laser module within thepattern-adjustable projector 110 may comprise a light emitter and someoptical components, where the light emitter may comprise avertical-cavity surface-emitting laser (VCSEL) array, for emittinginvisible light, and the optical components may comprise a diffractiveoptical element (DOE), a micro-lens array, a wafer-level optics (WLO)component, for generating the one or more predetermined patterns. Moreparticularly, the WLO component may comprise at least one wafer-levellens (e.g. one or more wafer-level lenses). In addition, thepattern-adjustable projector 110 may further comprise one or moreelectrically-controlled optical components such as at least one liquidcrystal (LC) lens (e.g. one or more LC lenses, any of which may bepositioned within or outside the laser module), for changing (e.g.tilting) one or more projection directions of the one or morepredetermined patterns.

For better comprehension, multiple light transmitting (Tx) paths fromthe pattern-adjustable projector 110 to the field 10 (e.g. invisiblelight Tx paths of invisible light from the pattern-adjustable projector110 to the field 10, and visible light Tx paths of strobe or torchemission from a light emitting diode within the ROI-based depthdetection device 100 to the field 10) may be collectively expressed withthe Tx direction (labeled “Tx” for brevity), and multiple lightreceiving (Rx) paths from the field 10 to the cameras 121 and 122 (e.g.invisible light Rx paths of invisible light from the one or more targetobjects 11 to the camera 121, and visible light Rx paths from the one ormore target objects 11 and the background thereof in the field 10 to thecamera 122) may be collectively expressed with the Rx direction (labeled“Rx” for brevity).

During the depth detection such as the ROI-based depth detection, theROI-based depth detection device 100 (e.g. the components therein) canperform the following operations:

(1) the camera 122 can capture a first image, where the first imagecomprises image contents indicating one or more objects (e.g. the one ormore target objects 11), and the first image typically does not havedepth information, and may represent a two-dimensional (2D) image;(2) the image processing circuit 130 (e.g. a first partial circuit ofthe image processing circuit 130, such as the image processor 132) candetermine a ROI of the first image (e.g. a region of a human face, suchas a region enclosing the face of the person mentioned above on thefirst image) according to the image contents of the first image, forfurther processing of the image processing circuit 130 (e.g. a secondpartial circuit thereof);(3) the image processing circuit 130 (e.g. the second partial circuit ofthe image processing circuit 130, such as the depth processor 131) canperform projection region selection to determine a selected projectionregion corresponding to the ROI among multiple predetermined projectionregions of the pattern-adjustable projector 110, where the selectedprojection region is selected from the multiple predetermined projectionregions according to the ROI;(4) the pattern-adjustable projector 110 can project a predeterminedpattern according to the selected projection region, for performing thedepth detection;(5) the camera 121 can capture a second image, where the second imagecomprises a projection result of the predetermined pattern on a firstobject (e.g. the human face such as the face of this person) among theone or more objects; and(6) the image processing circuit 130 (e.g. the depth processor 131) canperform the depth detection according to the second image to generate adepth map (e.g. the one or more depth maps such as the target depth map,or the at least one depth map such as the any depth map), where thedepth map indicates a set of depths of the first object.

For example, the first partial circuit of the image processing circuit130, such as the image processor 132, can generate ROI informationindicating the ROI, such as 2D ROI information indicating the ROI of the2D image (labeled “2D ROI” in FIG. 1 for better comprehension). Inaddition, the second partial circuit of the image processing circuit130, such as the depth processor 131, can obtain the ROI informationindicating the ROI from the first partial circuit such as the imageprocessor 132, and therefore can be notified of the ROI, for performingthe projection region selection as mentioned above. Additionally, theROI-based depth detection device 100 can utilize the image processingcircuit 130 (e.g. the depth processor 131) to generate one or morecontrol signals for controlling the pattern-adjustable projector 110,where the one or more control signals may carry selected projectionregion information indicating the selected projection region, fornotifying the pattern-adjustable projector 110 of the selectedprojection region. Under control of the image processing circuit 130(e.g. the depth processor 131), a location of the projection result ofthe predetermined pattern on the second image may correspond to alocation of the ROI on the first image. For example, the relativelocation of the projection result of the predetermined pattern withrespect to a reference point (e.g. a certain corner such as the upperleft corner) of the second image may be the same as the relativelocation of the ROI with respect to a corresponding reference point(e.g. a certain corner such as the upper left corner) of the firstimage.

As the ROI-based depth detection device 100 (e.g. the componentstherein, such as the pattern-adjustable projector 110, the imageprocessing circuit 130, the depth processor 131, etc.) can operateaccording to the method, the ROI-based depth detection device 100 canperform the ROI-based depth detection accurately and efficiently togenerate the one or more depth maps, such as the depth maps of the humanfaces, to make the electronic product equipped with the ROI-based depthdetection device 100 perform the 3D face recognition according to theone or more depth maps accurately and efficiently. As a result, theoverall performance can be enhanced.

According to some embodiments, the image processing circuit 130 (e.g.the depth processor 131) can calculate the set of depths according tothe difference between the projection result and the predeterminedpattern, where the projection result may indicate the variation of thepredetermined pattern due to non-planar surface of the human face. Moreparticularly, the predetermined pattern may comprise a plurality ofsub-patterns (e.g. a plurality of invisible-light tiles, etc.), andbased on the knowledge of the associated geometric relationships, theimage processing circuit 130 (e.g. the depth processor 131) cancalculate the set of depths according to the displacements of somesub-patterns (e.g. some invisible-light tiles) among the plurality ofsub-patterns (e.g. the plurality of invisible-light tiles, etc.) of thepredetermined pattern, since the depth variations in the set of depthsmay correspond to the displacements.

Regarding the depth calculations of the set of depths as describedabove, when a baseline between the center of the pattern-adjustableprojector 110 (e.g. the location of an optical axis of the opticalcomponents on a reference plane on the ROI-based depth detection device100) and the center of the image-capturing module 120 (e.g. an averageof the respective locations of respective optical axes of the cameras121 and 122 on the reference plane) is much shorter than the distancebetween the ROI-based depth detection device 100 and the one or moretarget objects 11 (e.g. the ratio of the length of the baseline to thisdistance is less than a predetermined ratio), the image processingcircuit 130 (e.g. the depth processor 131) can omit the baseline in thedepth calculations, where the length of the baseline can be forcibly setas zero, but the present invention is not limited thereto. In asituation where the baseline is considered, the image processing circuit130 (e.g. the depth processor 131) can use the real value of the lengthof the baseline in the depth calculations. For example, when both of thebaseline and a sub-baseline between the respective centers of thecameras 121 and 122 (e.g. the respective locations of the respectiveoptical axes of the cameras 121 and 122 on the reference plane) areconsidered, the image processing circuit 130 (e.g. the depth processor131) can use the real values of the respective lengths of the baselineand the sub-baseline in the depth calculations. For another example,when the baseline is considered but the sub-baseline is not considered,the image processing circuit 130 (e.g. the depth processor 131) can usethe real value of the length of the baseline and omit the sub-baselinein the depth calculations, where the length of the sub-baseline can beforcibly set as zero.

According to some embodiments, the predetermined pattern can beimplemented by way of structured light technology, etc., but the presentinvention is not limited thereto.

FIG. 2 illustrates some implementation details of the pattern-adjustableprojector 110 shown in FIG. 1 according to an embodiment of the presentinvention. The pattern-adjustable projector 110 can be implemented byway of various combinations of different sets of components, such asthat illustrated in Cases (a), (b), (c), (d), (e) and (f) shown in FIG.2 :

(a) the at least one LC lens comprises a LC lens positioned outside ofthe laser module, and the WLO component is positioned between theDOE/micro-lens array and the laser emitter;

(b) the at least one LC lens comprises a LC lens positioned outside ofthe laser module, and no WLO component is used;

(c) in a Type-1 configuration, the at least one LC lens comprises afirst LC lens positioned outside of the laser module and a second LClens positioned within the laser module, the DOE/micro-lens array ispositioned between the first LC lens and the second LC lens, and the WLOcomponent is positioned between the second LC lens and the laseremitter;(d) the at least one LC lens comprises a LC lens positioned within thelaser module, between the DOE/micro-lens array and the laser emitter,and no WLO component is used;(e) the at least one LC lens comprises a first LC lens positionedoutside of the laser module and a second LC lens positioned within thelaser module, between the DOE/micro-lens array and the laser emitter,and no WLO component is used; and(f) in a Type-2 configuration, the at least one LC lens comprises afirst LC lens positioned outside of the laser module and a second LClens positioned within the laser module, the DOE/micro-lens array ispositioned between the first LC lens and the WLO component, and thesecond LC lens is positioned between the WLO component and the laseremitter;but the present invention is not limited thereto. For example, as longas similar results can be achieved, some components may be integratedinto the same component and/or one or more components may be added,removed, or changed.

According to some embodiments, the laser emitter may comprise at leastone invisible-light source (e.g. one or more invisible-light sources),such as the VCSEL array, edge-emitting laser diodes, etc. In addition tothe at least one invisible-light source, the laser emitter may furthercomprise a beam limiting device (e.g. a collimator lens). The beamlimiting device can receive the emitted invisible-light from the atleast one invisible-light source and convert the emitted invisible-lightinto beam-limited invisible-light. For brevity, similar descriptions forthese embodiments are not repeated in detail here.

FIG. 3 illustrates, in the right half thereof, a small Tx field controlscheme of the method for performing ROI-based depth detection with aidof the pattern-adjustable projector according to an embodiment of thepresent invention, where a normal Tx field control scheme is alsoillustrated as shown the left half of FIG. 3 for better comprehension.The Rx field (e.g. the field 10 for the Rx direction) can be regarded asthe field of view (FOV), and can be referred to as the Rx FOV. Inaddition, the concept of the Tx field (e.g. the field 10 for the Txdirection) is similar to the concept of the FOV when omitting that theTx direction is opposite to the Rx direction. In some embodiments, theTx field (e.g. the field 10 for the Tx direction) can be referred to asthe Tx FOV for consistency, but the present invention is not limitedthereto.

As shown in the left half of FIG. 3 , assume that a non-adjustableprojector is capable of projecting its own pattern with a normal Txfield, and when projected, this pattern may distribute over the normalTx field (e.g. the whole of the field 10, if the non-adjustableprojector is arranged to project toward the field 10) in a corresponding3D raw image (labeled “3D Raw” for brevity). As shown in the right halfof FIG. 3 , the pattern-adjustable projector 110 can be designed toproject the predetermined pattern with a small Tx field (e.g. a Tx fieldnarrower than the normal Tx field), and can utilize the camera 121 tocapture a 3D raw image (labeled “3D Raw” for brevity), where theplurality of sub-patterns (e.g. the invisible-light spots) of thepredetermined pattern may distribute within the small Tx field (e.g. aportion of the field 10). In addition, the image processing circuit 130(e.g. the depth processor 131) can control the pattern-adjustableprojector 110 to selectively change the projection direction of thepredetermined pattern according to the ROI information. For example, theat least one LC lens in the pattern-adjustable projector 110 can bearranged to selectively change the projection direction of thepredetermined pattern according to the ROI information.

FIG. 4 illustrates a ROI-based projection control scheme of the methodaccording to an embodiment of the present invention. In Case (a) shownin the left half of FIG. 4 , the image processing circuit 130 (e.g. theimage processor 132) can utilize the camera 122 to capture a 2D image(labeled “2D” for brevity). Assume that no person appears in the field10, so the image processing circuit 130 (e.g. the image processor 132)can perform image processing such as dynamic range adjustment, imageenhancement, color adjustment, etc., as well as object detection, butdoes not detect any object since there is merely the background (labeled“without object detection” for brevity). In addition, the imageprocessing circuit 130 (e.g. the depth processor 131) can utilize thepattern-adjustable projector 110 to project the predetermined patternwith the small Tx field and utilize the camera 121 to capture a 3D rawimage (labeled “3D Raw” for brevity). As no object is detected, theimage processing circuit 130 (e.g. the depth processor 131) candetermine the selected projection region as a default projection region(e.g. a central projection region) among the multiple predeterminedprojection regions. As a result, the plurality of sub-patterns (e.g. theinvisible-light spots) of the predetermined pattern may distributewithin the default projection region.

In Case (b) shown in the right half of FIG. 4 , the image processingcircuit 130 (e.g. the image processor 132) can utilize the camera 122 tocapture a 2D image (labeled “2D” for brevity). Assume that the personappears in the field 10, so the image processing circuit 130 (e.g. theimage processor 132) can perform image processing such as dynamic rangeadjustment, image enhancement, color adjustment, etc., as well as objectdetection, and detect the target object such as the face of the person(labeled “with object detection” for brevity) and determine the ROI as aregion enclosing the face (labeled “2D Face ROI” for bettercomprehension). In addition, the image processing circuit 130 (e.g. thedepth processor 131) can utilize the pattern-adjustable projector 110 toproject the predetermined pattern with the small Tx field according tothe ROI and utilize the camera 121 to capture a 3D raw image (labeled“3D Raw” for brevity). As the target object such as the face of theperson is detected, the image processing circuit 130 (e.g. the depthprocessor 131) can determine the selected projection regioncorresponding to the ROI, where the selected projection region on thesecond image (the 3D raw image) has been moved from the default location(e.g. the central location of the central projection region) to thelocation corresponding to the ROI, such as the same relative location asthat of the ROI. As a result, the plurality of sub-patterns (e.g. theinvisible-light spots) of the predetermined pattern may distributewithin the selected projection region corresponding to the ROI.

For better comprehension, assume that the optical output power of the atleast one invisible-light source of the pattern-adjustable projector 110is the same as that of the invisible-light source(s) of thenon-adjustable projector. As the total invisible-light power of thepredetermined pattern is sent to the small Tx field that is narrowerthan the normal Tx field, the intensity of the predetermined pattern canbe greater than that of the pattern of the non-adjustable projector. Incomparison with the non-adjustable projector, the pattern-adjustableprojector 110 can generate the predetermined pattern to have higheraverage brightness in the small Tx field, to enhance at least one signalquality factor (e.g. signal-to-noise ratio (SNR), etc.) of theassociated processing in the Rx direction, as well as the image qualityof the second image (e.g. the 3D raw image), for enhancing the overallperformance. For brevity, similar descriptions for this embodiment arenot repeated in detail here.

FIG. 5 illustrates some examples of the multiple predeterminedprojection regions used in the method, where these examples correspondto 2D division. In a first case that the region count of the multiplepredetermined projection regions is equal to 4 (labeled “4 Regions” forbrevity), the depth detection device 100 (e.g. the pattern-adjustableprojector 110 and/or the depth processor 131) can control the multiplepredetermined projection regions as projection regions (e.g. equal-sizedprojection regions) corresponding to (2*2) sub-fields of the field 10;in a second case that the region count of the multiple predeterminedprojection regions is equal to 9 (labeled “9 Regions” for brevity), thedepth detection device 100 (e.g. the pattern-adjustable projector 110and/or the depth processor 131) can control the multiple predeterminedprojection regions as projection regions (e.g. equal-sized projectionregions) corresponding to (3*3) sub-fields of the field 10; and the restcan be deduced by analogy.

FIG. 6 illustrates some other examples of the multiple predeterminedprojection regions used in the method, where these examples correspondto one-dimensional (1D) division. In a first case that the region countof the multiple predetermined projection regions is equal to 2 (labeled“2 Regions” for brevity), the depth detection device 100 (e.g. thepattern-adjustable projector 110 and/or the depth processor 131) cancontrol the multiple predetermined projection regions as projectionregions (e.g. equal-sized projection regions) corresponding to 2sub-fields of the field 10; in a second case that the region count ofthe multiple predetermined projection regions is equal to 3 (labeled “3Regions” for brevity), the depth detection device 100 (e.g. thepattern-adjustable projector 110 and/or the depth processor 131) cancontrol the multiple predetermined projection regions as projectionregions (e.g. equal-sized projection regions) corresponding to 3sub-fields of the field 10, for example, horizontally divided orvertically divided; and the rest can be deduced by analogy.

FIG. 7 illustrates some examples of a location of an object (e.g. thetarget object) with respect to the selected projection region among themultiple predetermined projection regions used in the method. For bettercomprehension, assume that the selected projection region is positionedon the upper right corner of the field 10 in the first case shown inFIG. 5 . The person may appear in any of various places of the samecorner. The selected projection region is always valid for all of theassociated processing regarding this person.

FIG. 8 illustrates some examples of multiple transitional projectionregions serving as a portion of the multiple predetermined projectionregions used in the method. In addition to the examples in the firstcase shown in FIG. 5 , the examples that are added as shown in FIG. 8indicate the multiple transitional projection regions, and the extendversion of the multiple predetermined projection regions as shown inFIG. 8 (e.g. the original four predetermined projection regions in thefirst case shown in FIG. 5 and these transitional projection regions)can perfectly cover all possible locations of the person within thefield 10.

According to some embodiments, multiple reference 3D raw imagesrespectively corresponding to the multiple predetermined projectionregions can be generated (e.g. captured) in advance, for use ofdetermining the displacements mentioned above. For example, in asituation where the one or more objects 11 are replaced with apredetermined planar object such as a wall, a screen, etc. in alaboratory, the multiple reference 3D raw images may comprise multiplereference projection results of the predetermined pattern on thepredetermined planar object that correspond to the multiplepredetermined projection regions, respectively, where a reference 3D rawimage corresponding to a certain predetermined projection region amongthe multiple reference 3D raw images may comprise the projection resultcorresponding to this predetermined projection region. In addition, themultiple reference 3D raw images can be stored in a non-volatile memory(e.g. a Flash memory) within the image processing circuit 130 inadvance. When determining the selected projection region, the depthprocessor 131 (e.g. a pre-processing module therein) can select areference 3D raw image corresponding to the selected projection regionamong the multiple reference 3D raw images. As a result, the depthprocessor 131 (e.g. a depth decoder therein) can detect thedisplacements of some sub-patterns (e.g. some invisible-light tiles)among the plurality of sub-patterns (e.g. the plurality ofinvisible-light tiles, etc.) of the predetermined pattern, and calculatethe set of depths according to the displacements of these sub-patterns(e.g. these invisible-light tiles). For brevity, similar descriptionsfor these embodiments are not repeated in detail here.

FIG. 9 illustrates some implementation details of the depth processor131 and the pattern-adjustable projector 110 shown in FIG. 1 accordingto an embodiment of the present invention. The pattern-adjustableprojector 110 may further comprise one or more driving circuits such asa LC lens driver, for driving the at least one LC lens (labeled“Driving” for brevity). In addition, the depth processor 131 maycomprise the pre-processing module and the depth decoder, and thepre-processing module may comprise a region selection module and aground truth selection module.

The region selection module can obtain the ROI information indicatingthe ROI, such as the 2D ROI information indicating the ROI of the 2Dimage (labeled “2D ROI” in FIG. 9 for better comprehension), and performregion selection according to the ROI to determine the selectedprojection region corresponding to the ROI among multiple predeterminedprojection regions, and more particularly, generate the one or morecontrol signals for controlling the pattern-adjustable projector 110(e.g. the one or more driving circuits such as the LC lens driver). Asshown in FIG. 9 , the one or more control signals may comprise a regioninformation signal carrying the region information of the selectedprojection region (labeled “Region Info.” for brevity), for notifyingthe pattern-adjustable projector 110 (e.g. the one or more drivingcircuits such as the LC lens driver) of the selected projection region,and may further comprise a pulse-width modulation (PWM) signal (labeled“PWM” for brevity), for performing some other control. In addition, theground truth selection module can perform ground truth selection toselect a set of ground truth information corresponding to the selectedprojection region among multiple sets of predetermined ground truthinformation respectively corresponding to the multiple predeterminedprojection regions, and output the selected set of ground truthinformation (labeled “Ground truth” for brevity) to the depth decoder,for use of the depth calculations. Additionally, the depth decoder canobtain the selected set of ground truth information from the groundtruth selection module, and obtain the second image such as the 3D rawimage (labeled “Raw image” for brevity) from an image buffer within theimage processing circuit 130, where the image buffer can buffer (e.g.temporarily store) the second image received from the camera 121. Basedon the selected set of ground truth information, the depth decoder canperform depth decoding on the second image (e.g. the projection resultof the predetermined pattern on the first object, as recorded on thesecond image) to generate the depth map. For brevity, similardescriptions for this embodiment are not repeated in detail here.

In the embodiment shown in FIG. 9 , the one or more driving circuits canbe implemented within the pattern-adjustable projector 110, but thepresent invention is not limited thereto. In some embodiments, the oneor more driving circuits (e.g. the LC lens driver and the laser emitterdriver) can be implemented within the image processing circuit 130. Forbrevity, similar descriptions for these embodiments are not repeated indetail here.

According to some embodiments, the LC lens driver and the laser emitterdriver can be integrated into the same driver such as a hybrid driver,and the one or more driving circuits may comprise a single drivingcircuit such as hybrid driver, but the present invention is not limitedthereto. In some embodiments, the one or more driving circuits maycomprise multiple driving circuits such as the LC lens driver and thelaser emitter driver, where the LC lens driver and the laser emitterdriver are different drivers. For brevity, similar descriptions forthese embodiments are not repeated in detail here.

According to some embodiments, the depth decoder can performpredetermined mapping based on the selected set of ground truthinformation, such as ground truth mapping based on the selected set ofground truth information, to convert the displacements of somesub-patterns (e.g. some invisible-light tiles) among the plurality ofsub-patterns (e.g. the plurality of invisible-light tiles, etc.) of thepredetermined pattern into corresponding depths to complete the depthcalculations, but the present invention is not limited thereto. Inaddition, the multiple sets of predetermined ground truth informationmay comprise multiple sets of depth-calculation-related parametersrespectively corresponding to the multiple predetermined projectionregions, for use of performing predetermined mapping based on themultiple sets of predetermined ground truth information, respectively,where the multiple sets of depth-calculation-related parameters can begenerated according to the knowledge of the associated geometricrelationships in advance, but the present invention is not limitedthereto. For brevity, similar descriptions for these embodiments are notrepeated in detail here.

According to some embodiments, in a projection control procedure, thedepth processor 131 can control the pattern-adjustable projector 110(e.g. the one or more driving circuits such as the LC lens driver) todrive a certain LC lens (e.g. any LC lens among the at least one LClens) with signals corresponding to different polarities (e.g. apositive polarity and a negative polarity) alternately, to prevent acertain issue related to the LC lens. In addition, as the LC lens mayhave uncertain optical characteristics during transitions between thepolarities (e.g. the transition from the positive polarity to thenegative polarity, or the transition from the negative polarity to thepositive polarity), the depth processor 131 can control thepattern-adjustable projector 110 (e.g. the one or more driving circuitssuch as a laser emitter driver) to drive the laser emitter (e.g. the atleast one invisible-light source therein, such as the VCSEL array, theedge-emitting laser diodes, etc.) during a steady state period betweentwo adjacent transitions, to turn on the laser emitter (e.g. the atleast one invisible-light source therein) during the steady stateperiod, and more particularly, turn on the laser emitter (e.g. the atleast one invisible-light source therein) after entering the steadystate period and turn off the laser emitter (e.g. the at least oneinvisible-light source therein) before leaving the steady state period.For brevity, similar descriptions for these embodiments are not repeatedin detail here.

FIG. 10 illustrates a working flow of the method according to anembodiment of the present invention. The method may be applied to theassociated apparatus (e.g. the ROI-based depth detection device 100, theimage processing circuit 130, the depth processor 131, thepattern-adjustable projector 110, etc.), and may be executed by theapparatus.

In Step S11, the depth detection device 100 can utilize a first camerasuch as the camera 122 to capture the first image, where the first imagecomprises the image contents indicating the one or more objects such asthe one or more target objects 11.

In Step S12, the depth detection device 100 can utilize the imageprocessing circuit 130 (e.g. the first partial circuit of the imageprocessing circuit 130, such as the image processor 132) to determinethe ROI of the first image (e.g. the region of the human face) accordingto the image contents of the first image, for further processing of theimage processing circuit 130 (e.g. the second partial circuit thereof).For better comprehension, the ROI may represent the region where thehuman face is detected on the first image.

In Step S13, the depth detection device 100 can utilize the imageprocessing circuit 130 (e.g. the second partial circuit of the imageprocessing circuit 130, such as the depth processor 131) to perform theprojection region selection to determine the selected projection regioncorresponding to the ROI among the multiple predetermined projectionregions, where the selected projection region is selected from themultiple predetermined projection regions according to the ROI.

In Step S14, the depth detection device 100 can utilize thepattern-adjustable projector 110 to project the predetermined patternaccording to the selected projection region, for performing the depthdetection.

In Step S15, the depth detection device 100 can utilize a second camerasuch as the camera 121 to capture the second image, where the secondimage comprises the projection result of the predetermined pattern onthe first object (e.g. the human face) among the one or more objects.

In Step S16, the depth detection device 100 can utilize the imageprocessing circuit 130 (e.g. the depth processor 131) to perform thedepth detection according to the second image to generate the depth map(e.g. the one or more depth maps such as the target depth map, or the atleast one depth map such as the any depth map), where the depth mapindicates the set of depths of the first object.

In Step S17, the depth detection device 100 (e.g. the image processingcircuit 130) can determine whether to stop the working flow. If Yes, theworking flow comes to the end; if No, Step S11 is entered.

For example, in addition to the depth detection device 100, theelectronic product may further comprise a processing circuit forcontrolling operations of the electronic product, and the processingcircuit may comprise at least one application processor (e.g. one ormore application processors) for running various program codes such asan operating system (OS), drivers, application programs, etc. Undercontrol of a 3D face recognition application (App) running on the atleast one application processor, the processing circuit may send one ormore commands to the depth detection device 100 for controlling thedepth detection device 100. In response to a start command among the oneor more commands, the depth detection device 100 (e.g. the imageprocessing circuit 130) can start operating according to this workingflow, for returning the depth map mentioned in Step S16 to theprocessing circuit (e.g. the 3D face recognition App running on the atleast one application processor). In Step S17, the depth detectiondevice 100 (e.g. the image processing circuit 130) can check whether astop command among the one or more commands is received. When the stopcommand has been received from the processing circuit, the depthdetection device 100 (e.g. the image processing circuit 130) candetermine the checking result of Step S17 to be Yes. When no stopcommand has been received from the processing circuit, the depthdetection device 100 (e.g. the image processing circuit 130) candetermine the checking result of Step S17 to be No. For brevity, similardescriptions for this embodiment are not repeated in detail here.

For better comprehension, the method may be illustrated with the workingflow shown in FIG. 10 , but the present invention is not limitedthereto. According to some embodiments, one or more steps may be added,deleted, or changed in the working flow shown in FIG. 10 .

Those skilled in the art will readily observe that numerousmodifications and alterations of the device and method may be made whileretaining the teachings of the invention. Accordingly, the abovedisclosure should be construed as limited only by the metes and boundsof the appended claims.

What is claimed is:
 1. A method for performing region-of-interest (ROI)-based depth detection with aid of a pattern-adjustable projector, the method comprising: utilizing a first camera to capture a first image, wherein the first image comprises image contents indicating one or more objects; utilizing an image processing circuit to determine a ROI of the first image according to the image contents of the first image; utilizing the image processing circuit to perform projection region selection to determine a selected projection region corresponding to the ROI among multiple predetermined projection regions, wherein the selected projection region is selected from the multiple predetermined projection regions according to the ROI; utilizing the pattern-adjustable projector to project a predetermined pattern according to the selected projection region, for performing depth detection; utilizing a second camera to capture a second image, wherein the second image comprises a projection result of the predetermined pattern on a first object among the one or more objects; and performing the depth detection according to the second image to generate a depth map, wherein the depth map indicates a set of depths of the first object.
 2. The method of claim 1, wherein the first image does not have depth information.
 3. The method of claim 1, wherein the first image represents a two-dimensional image.
 4. The method of claim 1, wherein utilizing the image processing circuit to determine the ROI of the first image according to the image contents of the first image further comprises: utilizing a first partial circuit of the image processing circuit to determine the ROI of the first image according to the image contents of the first image, for further processing of a second partial circuit of the image processing circuit.
 5. The method of claim 4, wherein the first partial circuit of the image processing circuit generates ROI information indicating the ROI; and utilizing the image processing circuit to perform the projection region selection to determine the selected projection region corresponding to the ROI among the multiple predetermined projection regions further comprises: utilizing the second partial circuit of the image processing circuit to obtain the ROI information indicating the ROI from the first partial circuit, and to perform the projection region selection to determine the selected projection region corresponding to the ROI among the multiple predetermined projection regions.
 6. The method of claim 1, further comprising: utilizing the image processing circuit to generate one or more control signals for controlling the pattern-adjustable projector, wherein the one or more control signals carry selected projection region information indicating the selected projection region, for notifying the pattern-adjustable projector of the selected projection region.
 7. The method of claim 1, wherein a location of the projection result of the predetermined pattern on the second image corresponds to a location of the ROI on the first image.
 8. The pattern-adjustable projector that operates according to the method of claim
 1. 9. The image processing circuit that operates according to the method of claim
 1. 10. A depth processor that operates according to the method of claim 1, wherein a first partial circuit of the image processing circuit determines the ROI according to the image contents of the first image, and the depth processor is a second partial circuit of the image processing circuit and performs the projection region selection to determine the selected projection region corresponding to the ROI among the multiple predetermined projection regions.
 11. An apparatus for performing region-of-interest (ROI)-based depth detection, the apparatus comprising: a first camera, arranged to capture a first image, wherein the first image comprises image contents indicating one or more objects; an image processing circuit, coupled to the first camera, arranged to determine a ROI of the first image according to the image contents of the first image, and perform projection region selection to determine a selected projection region corresponding to the ROI among multiple predetermined projection regions, wherein the selected projection region is selected from the multiple predetermined projection regions according to the ROI; a pattern-adjustable projector, coupled to the image processing circuit, arranged to project a predetermined pattern according to the selected projection region, for performing depth detection; and a second camera, coupled to the image processing circuit, arranged to capture a second image, wherein the second image comprises a projection result of the predetermined pattern on a first object among the one or more objects; wherein the image processing circuit performs the depth detection according to the second image to generate a depth map, wherein the depth map indicates a set of depths of the first object. 